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ABSTRACT 



Twenty-seven seventh-grade students participated in 
a study designed to investigate the student rate of progress through 
the computer-assisted instruction (CAI) text in relation to ability 
and achievement. Measures were developed to determine the general 
reading rate for each individual and to notice any deviation from 
that rate. It was found that word rate and word rate stability as 
measures of reading behavior predict science achievement with greater 
efficiency than the Metropolitan Achievement Test (MAT) for reading. 
The word rate measure also appears to have more predictive power than 
latency. Appendices contain statistical data collected by the study 
and a reference list. (JY) 
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FOREWORD 



The ISCS Technical Report Series is intended to provide ccmnunication 
to other colleagues and interested professionals who are actively interested 
in research with and development of curriculum material. The rationale for 
the Technical Report series is three-fold. First, to report in a concise, 
descriptive, and explanatory nature advances made in the technology of 
curriculum development . Second, to give quick distribution to pilot studies 
that show great premise with potential for further research and subsequent 
reporting. Third, to provide for distribution of pre-publication copies of 
inplementatlon studies that, after proper technical review, will ultimately 
be found in professional journals. 



This report suggests a premising new way to monitor student progress 
througn printed curriculum material with irrpli cations for the evaluation of 
students in terms of achievement . The word-rate measure reported also has 
potential for researchers interested in investigating the reading behavior of 
students . 



Ernest Burkinan, Director 
Intermediate Science Curriculum Study 



July 20, 1970 

The Florida State University 
Tallahassee, Florida 
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GENERAL BACKGROUND ON THE EflERMEDIATB SCIENCE CURRICULUM STUDY 

The Intermediate Science Curriculum Study (ISCS) is a large-scale instruc- 
tional research project supported to date by a contract with the United 
States Office of Education and grants from the National Science Foundation. 
The project is designed to develop, test, and disseminate into practice 
a system of individualized science instruction for grades seven through 
nine. 

The project is organized on a develop -field-test-re vise desigi. Draft 
materials are produced at Florida State University by on- and invited 
off- campus personnel and tested on a large national sample of junior-high- 
school students. During the 1969-70 school year, more than 75,000 students 
in 25 states are involved in the field testing of the ISCS materials. In 
addition, a snail number of students from the Florida State University 
canpus school are taking a computer-assisted instruction version of the 
materials from which additional feedback data are being accumulated'. To 
date, more than 400 scientists, teachers, and education specialists have 
cooperated in the development process. 

The most unique feature of the ISCS materials is the fact that the students 
using them progress at different rates and through different instructional 
pathways depending upon their interests, abilities, and previous experiences . 
The materials are being designed that tMs can be accomplished in ordinary 
science classrooms by teacher's with limited special training. 

The package of instructional materials for each grade level consists of 
student printed materials, specially designed laboratory apparatus, a 
student self-evaluation system based upon behavioral objectives established 
for the instructional materials, teacher orientation materials, and 
standardized tests. The Silver Burdett Corporation , in conjunction with 
Damon Educational Corporation, is distributing these materials during the 
experimental phase of the project and will market the commercial version 
of them. 

The project has generated world-wide interest ; and its newsletter, 
published twice yearly, new goes to more than 10,000 people in 42 countries. 
ISCS materials are new in use in Australia and will be used in American 
dependent schools in Germany and Japan in September. Experimental testing 
of the materials is now underway in Manila, and plans have been established 
for a joint Florida State University - Philippines effort to produce a 
special Philippines version of the program. In addition, project personnel 
have visited Japan, India, and several South American countries for 
preliminary discussions related to possible use of the materials in these 
areas. 
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CAI WORD RATE - STUDEWT ABILITY AS PREDICTORS OF ACHIEVb3yM\lT 
Introduction 

In the fall of 1966, the Intermediate Science Curriculum Study (ISCS) 
began a large scale curriculum effort in which Computer Assisted Instruction 
(CAI) was envisioned as an economically defensible and uniquely valuable 
research and development tool for use by the curriculum developer. The 
application of CAI held promise as an appropriate and effective means of 
monitoring in detail individual student progress and performance in laboratory- 
centered, self-pacing printed student materials , Two and one -half grade 
levels of ISCS science materials have been programmed f or CAI presentation 
cn an IBM 1500 system. ISCS Technical Report (Teates, et al. , 1970) describes 
the ways in which CAI information has been programmed and analyzed. This 
report is concerned with the student rate of progress through the CAI text 
in relation to ability and achievement . 

Background Information 

In tire final analysis, the two viable learner resources most readily 
available to the curriculum develqrer are student ability and student time. 
Because of the availability of latency and student-response information, 

CAI provides a straight -forward means of examining, for any student, the 
time spent on any frame of material as well as his achievement on that frame. 
Correct responding is an obvious variable that has a high probability of 
predicting successful terminal performance. Latency, cn the other hand, 
is a less well understood, though often measured, learning variable. Latency 
is defined as the interval between the presentation of a unit of instruction 
to the learner and the learner's response to the unit. The unit may be 
carposed of information or a question which calls for an overt response 
on the part of the learner. 
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Suppes' (196*1) suggested that latency may be more sensitive than errors 
in revealing mastery and depth and thus be an excellent learning criterion. 
Latency has the advantages of providing a trial-to-trial measure of learner 
performance and of always being available. Brror rate, for example, is only 
available when the learner is forced into making an overt response. Time 
data exist whether the learner responds overtly or covertly (thinks his answer) 

According to Brooks (1961), when both error rate and latency are avail- 
able for individuals, they are likely to be significantly correlated with 
one another. Brooks further pointed out that time data, although having the 
advantage of being recorded automatically, have limitations in their usefulness 
Hie correlation of time data with other variables relating to the quality 
of student learning is poorly understood (Brooks and Clark, 1969). 

lime data were used in a study involving individually prescribed instruc- 
tion (IPI) by Yeager and Kissel (1969). It was found that the number of days 
required to master a given unit in mathematics was related to the learner's 
initial entering state. Hie most inportant factors of initial learner state 
were the learner's pre-test score, number of skills to master, and learner's 
age. 

In a study by Dick and Latta (1969) comparing the effects of ability 
and mode of instruction (CAI and PI), it was found that low ability students 
spent increasing time in the actual production of correct answers but did 
not perform significantly better. Low ability students spent more time on 
CAI , made significantly more errors , and had significantly lower terminal- 
test performance as compared to low ability students using programmed instruc- 
tion . 
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Brooks and Clark (1969) conducted a study with junior-college students. 
Hie students were given a test, and latency data were recorded for each item. 
Bach student then reviewed test items selected on the basis of the time he 
had spent responding to the items. The students were divided into two groups 
for the study. One group reviewed items they had answered slowly. For each 
student, slow answering on each item was defined as responding in more than 
his own median time for all test items. The other group renewed their 
quickly answered items . When retested, the students who had reviewed their 
slowly answered items made fewer errors on the re-test . They also showed 
a significantly greater reduction in percentage of errors from first to 
second re-test. 

In replication studies by Brooks, Clark, and Brown (1969) de alin g with 
learning as a function of the relationship between student response time and 
difficulty of instructional items, it was found that subjects made fewer 
post-test errors when spending more response time on difficult (relative to 
easy) instructional items . It was proposed that learning may be increased 
by causing students to change the distribution of time on learning tasks by 
causing them to better match time spent with task difficulty. 

These studies tend to support the idea that tine spent on learning is 
an interesting variable which needs further study. The extent to which this 
variable correlates with student entering ability, error rate, and general 
abilities may be of special value when attempting to provide for individual 
differences among learners. 

Purpose 

This report describes inter-correlations for the general abilities of 
intelligence and reading, specific ISCS achievement in terms of both frame 
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responses and post instruction achievement test scores, and latency. Hie 
purpose of this study is to examine the relative extent to which latency, 
and quantities derived from it, can be used to predict achievement. 

Procedure 

Twenty seventh-grade students were selected from the Florida State 
University School to participate in the ISCS CAI classroom for the 1968-69 
school year. These students were selected to provide subjects of a broad 
range of abilities. The California Test of Mental Maturity (CIMM, Level 3) 
and the Metropolitan Achievement Test of Reading (MAT Reading, Form AM) were 
administered to the group selected. The Alternate (Form BM) of the MAT 
Reading test and the ISCS science achievement test (1968 experimental edition) 
were administered to the same students as post-tests. The raw score means 
and standard deviations for the students on these tests are shown in Table I. 
No claim is made that the distribution of the 20 scores is normal. 

CHARACTERISTICS OF THE ISCS-CAI STUDENT GROUPS 



TEST 


MEAN 


s .d. 


RANGE 


N 


CTMM 


78.05 


10.65 


61-98 


20 


MAT READING (pre-test) 


29.75 


7.52 


10-41 


20 


MAT READING (post-test) 


30.11 


5.70 


15-36 


18 


ISCS ACHIEVEMENT 


27.55 


5.29 


18-38 


20 


TABLE I 



Twenty-two chapters of ISCS seventh-grade core materials were programmed. 

From these materials, four chapters were selected. The criteria for selections 




were: 
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1. Sampling of student behavior over time. 

2. Sampling of chapters which contained material representative of a 
variety of frame types . 

The four types of CAI frames selected were: 

1. Text material preceded and followed by frames of the same typn. 

2. Questions which require answers directly related to text frames. 

3. Activity frames where student latency included time to complete 
specified activities . 

4. Text material containing a rhetorical question. 

Ten students were randomly selected from the group of twenty students 
in the class. Tire means, standard deviation, and standard errors of measure- 
ment for the sample are shown in Table II. Using the "t" test, with finite 
population correction, no significant differences were found at the .05 . 
confidence level between the standardized test mean scores for this sample 
and those of the population reported in Table I. 

DESCRIPTIVE STATISTICS OP THE TEN CAI STUDENT SAMPLES 



TEST 


MEAN 


s .d. 


SEjyj 


CTMM 


74.7 


11.17 


3.53 


MAT READING (pre-test) 


2?. 8 


7.30 


2.31 


MAT READING (post-test) 


2?.l 


6.03 


1.91 


ISCS ACHIEVEMENT 


26.8 


6.47 


2.05 




TABLE II 
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For purposes of this study, it was felt that the mean latency per word 
of text would be a useful means of treating variations in frame length. 
Word-rate, as used in this study, is defined as the ratio of the number of 
words per frame to the latency for that frame and is expressed in units of 
words per minute. With such a measure, it becomes possible to determine 
readily the general reading rate for each individual student and to notice 
any deviations from that rate. Further, the relation of such deviations to 
frame responses and general course adiievement can be readily examined. 

The procedure used to get these word-rate data began by identifying the 
particular frames for which latency was not likely affected by student reaction 
to an adjacent frame i.e., only those frames which required no response such 
as carrying out instructions or answering questions . Once identified, the 
frame latencies were extracted for each student using analysis programs 
developed by ISGS (Teates, et al . , 1970). The word counts for each frame 
were determined and the word rates correlated. In all, 43 frames were 
selected from Chapters 3, 10, 15, and 19 for text frames. The number of 
rhetorical question frames selected for word-rate calculations was nine. 

Because rhetorical question frames were not used in Chapters 15 and 19, Chap- 
ter 8 frames were substituted to add to those from Chapters 3 and 10. 

Results and Discussion 

All examination of student word-rate response for the frames selected 
showed considerable variability of word rate around the student's mean word 
rate within each chapter. Furthermore, consistent large variability was 
observed for certain students but not for others. For purposes of analysis, 
a derived variable was calculated for each student as a potential correlate 
of his performance. This variable, which can be thought of as word rate 
stability, is defined as: 

O 




Word rate stability = 



7 

V 

L 



| ( frame word rate - ire an word rate) 



Number of Frames 

Note the WR stability is always a positive quantity by this definition as 
it is defined as the average absolute magnitude of the difference between 
the word rate of a student on a particular frame less his average word rate 
for all frames. 

Because data for a snail sample number of CAT students were used in 
this study and because no assumption of normality is claimed regarding the 
distribution of test scores for these students , the Spearman Rank Order 
correlation was selected to estimate tire inter-relationship of achievenent, 
ability, and CAI variables. The eleven variables used were: 

1. General intelligence as measured by the CTMM, Level 3. 

2. Entering reading ability as measured by the MAT Reading Test, 



form AM. 



3. End of course reading ability as measured by the MAT Reading Test, 
form BM. 

4. End of course science achievement as measured by the ISCS achieve- 
ment test. 

5. Word rate as measured in Chapters 3, 10 3 15, and 19. This word 
rate is the mean rate for all four chapters expressed in words/seccnd. 

6. Total time or latency in Chapters 3, 10, 15, and 19 combined 
expressed in minutes . 

7. Word rate stability within Chapters 3, 10, 15, and 19 . This is 
the quantity which is really the unsigned accumulation of all the deviations 
frcm the mean word rate for each frame. 




8. Total time or latency .in Activity Frames for Chapters 3, 8, 10, 

and 19 contoined expressed in minutes. This measure includes all time expended 
in reading the instructions and carrying out the laboratory activities. 

9. Word rate for rhetorical question text frames for Chapters 3, 8, 
and 10 expressed in words/sec. 

10. Total time or latency in responding to questions in Chapters 3, 8, 

10, and 19 expressed in minutes. 

11. Number of correct responses to questions in Chapters 3 , 8, 10, and 
19. 

The correlation matrix for these eleven variables is given in Table III 
below. Appendix 1 lists the values of each of these variables for each 
student . 



Spearman Rank Inter-Correlation® of Achievement* 
Ability, and CAI Variable* 



Variable 




1 


2 


3 


4 


5 


6 


‘7 


8 


9 10 


1 


CTMM ■ 


0 . ooo 


















2 


Reading (pre-test) 


. 655 * 


0*000 
















3 


Reading (postrteat) 


.497 




0.000 














4 


ISCS Ach. 


. 752* 


,624 


.479 


0,000 












5 


Word Rate lor Text 


736* 


~. £ 24 ** 


824 ** 


700 * 


0.000 










6 


Total Time in Ch, 


.342 


. 194 


,624 


. 439 * 


-.576 


0 * 000 








7 


Word Rate Stability 


-. 773** 


-. 667 * 


-. 758 ** 


-. 705 ** 


. 867 ** 


758 * 


0.000 






a 


Activity Frame Time 


-.045 


-.491 


155 


155 


.079 


.358 


.018 


0.000 




9 


W. R. for Rhet. Qu. 


-.455 


-.391 


-.552 


-.585 


, 630 * 


-» B 03 ** 


, 785 ** 


.033 


0.000 


10 


Qu. Response Time 


.009 


.091 


,455 


.221 


-.430 


, 794 ** 


-,503 


. 176 


-. 688 * 0.000 


11 


No Correct Qu* Resp. 


.070 


. 130 


,327 


.442 


-.079 


. 491 ' 


-,255 


133 


-.503 .327 



* sig at . 05 level 
** sig at . 01 level 



o 




table nr 
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Eighteen statistically significant relationships appear in this matrix. 
Fifteen of these relationships directly involve the word rate in text material, 
word rate for rhetorical questions, and word rate stability measures. The 
other three correlations are for pre- and post-test MAT reading (.806), CTMM 
with pre-test reading (.655), and CTMM with ISCS achievement (.752). 

The word rate and word rate stability measures have higher correlations 
(-.700 and -.785) with ISCS achievement than either MAT reading pre- or 
post-test (.624 and .479). The reduction in reading correlation from pre- to 
post-test may be interpreted in the light of studies that have consistently 
suggested that reading ability of seventh-grade ISCS students has improved 
10 to 12 percentile rank points over and above maturation (ISCS, 1970). The 
thesis is that, as the student's general reading skills improve, his science 
achievement becomes less predictable by these skills. The students' rank 
order of improvement in reading differs from that of their science achievement . 
This is further supported by the reduction in reading pre- and post-test 
correlations with general intelligence (.655 and .497) and both the previously 
mentioned high correlations of both intelligence with ISCS achievement and 
pre- and post-test reading intercorrelation. 

Word-rate highly correlates with the general reading skills (-.824 and 
-.824) as does the word rate stability (-.667 and -.758). These measures 
exhibit consistent significant correlations as the students progress through 
the course as is shown in Table IV. They also retain high correlations to 
general intelligence (-.736 and -.773). Interpretations of these correlations 
support the thesis that both word rate and word rate stability do serve the 
role of a significant measure of reading behavior and are important predictors 
of content achievement. 
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CORRELATIONS OP WORD RATE AND WORD RATE STABILITY 
WITH ISCS ACHIEVEMENT BY CHAPTER 





Ch. 3 


Ch. 10 


Ch. 15 


Ch. 19 


Word Rate 


-.348 


-.670 


-.758 


-.645 


Word Rate Stability 


-.482 


-.785 


-.785 


-.682 



TABLE IV 

The negative sign associated with both the word rate and word rate 



stability measure correlations requires explanation. One might suspect 
that the higher achiever reads more slowly but at a more uniform rate. Such 
an interpretation is misleading. The data do suggest that the student who 
progresses at a uniform reading rate has high intelligence and attains a 
higher ISCS achievement test score than the student with an erratic reading 
rate. However, the high mean word rate of the low achiever may result from 
his skipping through the reading passages, thus not reading them at all. 

This interpretation also leads one to suspect that the poor reader could be 
expected to complete each chapter (and thus the course) before the high 
achiever. The correlation of total tine in chapters with word rate (-.576) 
supports this . However, the positive correlation ( . 358) of word rate with 
activity frame time suggests that the poor reader also takes longer to carry 
out the instructions he was meant to read in the frame, thereby retarding; 
his rate of progress through the curriculum material. Conversly, the poor 
reader tends to take less time to respond to chapter questions (r = -.43), 
and has less success in answering them (r = -.503). 

Multiple correlations between the criterion ISCS Achievement test s cores 
and the word rate, reading and intelligence measures were computed by means 
of step wise regression. The coefficients are listed in Table V and were 

obtained by confining the test scores in the order of their contribution. 

O 
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Table VI lists similar coefficients which were obtained when the regression 
was determined using word rate stability in lieu of the word rate data. The 
stepwise regression computer program (B1C02R) contained in the BID Computer 
Program Manual (University of California Press, 1967) was in the computations. 

SUMMARY TABLE 

MULTIPLE REGRESSION ANALYSIS 
USING WORD RATE 



Step Variable Multiple p Increase F No. of Dep. 

Number Entered R R in R 2 Value Var. incl. 



1 


CTMM 


.6981 


.4873 


.4873 


7.6042 


1 


2 


Word Rate 


.7875 


.6202 


.1328 


2.4479 


2 


3 


Reading (pre) 


.7924 


.6279 


.0077 


.1248 


3 


4 


Reading (post) 


• 7949 


.6316 


.0037 


.0499 


4 



TABLE V 



SUMMARY TABLE 

MULTIPLE REGRESSION ANALYSIS 
USING WORD RATE STABILITY 



Step Variable Multiple Increase F No. of Dep. 

Number Entered g B 2 in r2 Value Jfeaa . incl 



1 


CTMM 


.6981 


.4873 


.4873 


7.6042 


1 


2 


WR Stability 


.7947 


.6315 


.1442 


2.7385 


2 


3 


Reading (pre) 


.7980 


.6368 


.0054 


.0886 


3 


4 


Reading (post) 


P- level 


insufficient 


for computation 
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TABLE VI 
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Tables V and VI show that General Intelligence makes most significant 
contribution to the regression equation, followed by the word rate or word 
rate stability measures to account for better than 62# of the variance in the 
criterion achievement test scores. The contribution of the reading test scores 
in both instances is relatively small as predictor variables. 

Rhetorical question frames were employed by the ISCS authors for tne pur- 
pose of focusing student attention and provoking thought . Word rates were 
calculated for such frames in Chapters 3, 10, and 19 and found to exhibit 
significant correlations with the text materials word rate (.63), total time 
in chapter (-.803), and word rate stability (.785). In addition, these were 
moderate, but not’ significant, correlations with both ISCS achievement (-.583) 
and the number of correct question responses (-.503). 

Summary 

This study gives strong evidence that word rate and word rate stability 
as measures of reading behavior predict science achievement with greater 
efficiency than the MAT general reading tests . Further, the predictive 
efficiency remains high throughout the course in contrast to the standardized 
reading test which loses predictive power as the course progresses. To the 
extent that the ISCS achievement test or the student's responses to chapter 
questions can be regarded a valid criterion measure, the word rate measure 
appears to have more predictive power than latency alone as suggested by Suppes 
(1964). 

The findings reported in this study are consistent with the statement 
of Brooks and Clark (1969) that correlation of time data with other variables 
relating to quality of learning is poorly understood. Word-rate and stability 
measures may measure students' involvement with or tracking of the printed 
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word, and as such be related to the quality of conmunication between the author 
and learner. To the extent this is true, the CAI word rate measures can prove 
to be invaluable tools in the formative evaluation of curriculum materials. 

This matter deserves further research. 
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Data Listing for Eleven Variable 
on 10 CAI Students 
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Variable 

Wo. Description 



1 CTMM 

2 MAT Be aiding (pre) 

3 MAT Reading (post) 

4 ISCS Achievement 

5 Word Rate, Ch. 3 s 

6 Word Rate, Ch 

7 Word Rate, Ch 

8 Word Rate, Ch 
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752 755 756 759 760 761 762 764 



10 * 
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30 
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28 
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29 
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21 


18 


31 


34 


22 


16 
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13 
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16 
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68 


18 


60 


29 


29 111 


12 


Tine in Ch. 19 




246 


100 154 


85 118 234 


13 


W.R. Stability, Ch. 3“ 




4 


119 


7 


27 105 


8 


14 
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Spearman Rank Order Correlation Matrix 
of 

Intelligence^ Ability and Achievement Variables 

with 

CAI Variables by Chapter 
N = 10 Subjects 





Variable 


12 3 


4 5 6 7 


8 


9 


1 


CTMM 


0.000 .655 .497 


.752 -.476 -.355 -.809 


-.561 


.221 


2 


MAT Reading (pre) 


0.000 .806 


.624 -.533 -.582 -.406 


-.379 


.109 


3 


MAT Reading (post) 


0.000 


.479 -.600 -.727 -.397 


-.585 


.473 


4 


ISCS Achievement 


0.000 -.348 -.670 -.758 


-.645 


.191 


5 


Word Rate, Ch. 3 




0.000 .321 .258 


.300 


.067 


6 


Word Rate, Ch. 10 




0.000 .439 


.452 


-.455 


7 


Word Rate, Ch. 15 




0.000 


.667 


-.445 


8 


Word Rate, Ch. 19 






0.000 


-.427 


9 


Time in Ch. 3 








0.000 
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Variable 



1 


CIMM 




2 


MAT Reading (pre) 




3 


MAT Reading (post) 
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ISCS Achievement 
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Word Rate, Ch. 3 
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Word Rate, Ch. 10 
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W.R. Stability, Ch. 


10 


15 
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Variable 

1 CTMM 

2 MAT Reading (pre) 

3 MAT Reading (post) 

4 ISCS Achievement 

5 Word Rate j Ch. 3 

6 Word Rate, Ch. 10 

7 Word Rate, Ch. 15 

8 Word Rate, Ch. 19 

9 Time in Ch. 3 

10 Time in Ch. 10 

11 Time in Ch. 15 

12 Time in Ch. 19 

19 Qu. Response Time, Ch. 3 0. 

20 Qu. Response Time, Ch. 8 

21 Qu. Response Time,, Ch. 10 

22 Qu. Response Time, Ch. 19 

23 Correct Responses, Ch. 3 

24 Correct Responses, Ch. 8 

25 Correct Responses, Ch. 10 

26 Correct Responses, Ch. 19 
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